Appendix: Methods for the Classification of Data from Open-Ended Questions in Surveys

Disputation
16 April 2024

Camille Landesvatter

University of Mannheim

Appendix Intro

Overview of Methods for Studies 1-3

Table 1: Overview Classification Methods and Examples. Source: Own depiction.

Characteristics of Open-Ended Survey Answers

Figure 1: The previous question was: ‘How often can you trust the federal government in Washington to do what is right?’. Your answer was: ‘[Always; Most of the time; About half of the time; Some of the time; Never; Don’t Know]’. In your own words, please explain why you selected this answer.

Appendix Study 1

Study 1: Results

Figure 1: Distribution of associations with known people across trust measures. Note: CIs are 95%, n=7,497.

Study 1: Results

Figure 2: Distribution of associations and their sentiment across trust measures. Note: CIs are 95%.

Study 1: Results

Figure 3: Associations and trust scores across different measures. Note: CIs are 95% and 90%.

Appendix Study 2

Study 2: Item Nonresponse by Condition

Figure 5: Item response rates by experimental condition and item.

Study 2: Sample Composition (Sample ↔︎ Population)

Table 1: Sample Characteristics in Comparison to US Census Data (2015).

Study 2: Covariate Balance (Text and voice)

Table 2: Covariate Balance between Text and Voice Condition.

Study 2: Results

Figure 6: Information Content Measures across Questions.
Note. CIs are 95%, n_vote-choice: 830 (audio: 225, text: 605), n_future-children: 1,337 (audio: 389, text: 748)

Study 2: Results

Figure 6: Exemplary Survey Answers by Entropy.

Appendix Study 3

Study 3: Results

Figure 7: Sentiment Classification with three categories by classifier (BERT vs. GPT).
Note. n=491 open-ended answers.

Study 3: Results

Figure 8: Linear model of sentiment and a five-category trust score (bi- and multivariate).
Note. n=491 open-ended answers, GPT classification.

Study 3: Results

Figure 9: Sentiment Classification with five categories by classifier (AFINN vs. VADER).
Note. n=496 open-ended answers.

Study 3: Results

Figure 9: Emotion Classification obtained from EmoLex.
Note. n=404 open-ended answers, multi-class assignment.

Appendix: Methods for the Classification of Data from Open-Ended Questions in Surveys